Hidden Markov Models and their Application for Predicting Failure Events

We show how Markov mixed membership models (MMMM) can be used to predict the degradation of assets. We model the degradation path of individual assets, to predict overall failure rates. Instead of a separate distribution for each hidden state, we use hierarchical mixtures of distributions in the exponential family. In our approach the observation distribution of the states is a finite mixture distribution of a small set of (simpler) distributions shared across all states. Using tied-mixture observation distributions offers several advantages. The mixtures act as a regularization for typically very sparse problems, and they reduce the computational effort for the learning algorithm since there are fewer distributions to be found. Using shared mixtures enables sharing of statistical strength between the Markov states and thus transfer learning. We determine for individual assets the trade-off between the risk of failure and extended operating hours by combining a MMMM with a partially observable Markov decision process (POMDP) to dynamically optimize the policy for when and how to maintain the asset.Comment: Will be published in the proceedings of ICCS 2020; @Booklet{EasyChair:3183, author = {Paul Hofmann and Zaid Tashman}, title = {Hidden Markov Models and their Application for Predicting Failure Events}, howpublished = {EasyChair Preprint no. 3183}, year = {EasyChair, 2020}

An Approximate Dynamic Programming Approach to Urban Freight Distribution with Batch Arrivals

We study an extension of the delivery dispatching problem (DDP) with time windows, applied on LTL orders arriving at an urban consolidation center. Order properties (e.g., destination, size, dispatch window) may be highly varying, and directly distributing an incoming order batch may yield high costs. Instead, the hub operator may wait to consolidate with future arrivals. A consolidation policy is required to decide which orders to ship and which orders to hold. We model the dispatching problem as a Markov decision problem. Dynamic Programming (DP) is applied to solve toy-sized instances to optimality. For larger instances, we propose an Approximate Dynamic Programming (ADP) approach. Through numerical experiments, we show that ADP closely approximates the optimal values for small instances, and outperforms two myopic benchmark policies for larger instances. We contribute to literature by (i) formulating a DDP with dispatch windows and (ii) proposing an approach to solve this DDP

Dynamic programming with approximation function for nurse scheduling

Although dynamic programming could ideally solve any combinatorial optimization problem, the curse of dimensionality of the search space seriously limits its application to large optimization problems. For example, only few papers in the literature have reported the application of dynamic programming to workforce scheduling problems. This paper investigates approximate dynamic programming to tackle nurse scheduling problems of size that dynamic programming cannot tackle in practice. Nurse scheduling is one of the problems within workforce scheduling that has been tackled with a considerable number of algorithms particularly meta-heuristics. Experimental results indicate that approximate dynamic programming is a suitable method to solve this problem effectively

Bayesian Best-Arm Identification for Selecting Influenza Mitigation Strategies

Pandemic influenza has the epidemic potential to kill millions of people. While various preventive measures exist (i.a., vaccination and school closures), deciding on strategies that lead to their most effective and efficient use remains challenging. To this end, individual-based epidemiological models are essential to assist decision makers in determining the best strategy to curb epidemic spread. However, individual-based models are computationally intensive and it is therefore pivotal to identify the optimal strategy using a minimal amount of model evaluations. Additionally, as epidemiological modeling experiments need to be planned, a computational budget needs to be specified a priori. Consequently, we present a new sampling technique to optimize the evaluation of preventive strategies using fixed budget best-arm identification algorithms. We use epidemiological modeling theory to derive knowledge about the reward distribution which we exploit using Bayesian best-arm identification algorithms (i.e., Top-two Thompson sampling and BayesGap). We evaluate these algorithms in a realistic experimental setting and demonstrate that it is possible to identify the optimal strategy using only a limited number of model evaluations, i.e., 2-to-3 times faster compared to the uniform sampling method, the predominant technique used for epidemiological decision making in the literature. Finally, we contribute and evaluate a statistic for Top-two Thompson sampling to inform the decision makers about the confidence of an arm recommendation

Sometimes You Cannot Have It All: Party Switching and Affiliation Motivations as Substitutes

Existing research on when legislators switch parties reports inconsistent results about motivations for switching (e.g., office, ideology, and votes). I treat the motivations for party switching as substitutes and argue that many of the inconsistencies that persist can be explained by modelling the interactive effects between these motivations. For example, scholars differ in terms of whether they find that electoral considerations are an important determinant of party switching. The conflicting findings on the independent effects of electoral considerations are explained here by demonstrating that these effects are conditional on the level of office benefits a legislators enjoys, as well as the ideological distance between the legislator and party. More generally, the empirical analysis provides strong support for the substitution effect hypothesis. Thus, modelling interactive effects increases our understanding of party switching

Lookahead policy and genetic algorithm for solving nurse rostering problems

Previous research has shown that value function approximation in dynamic programming does not perform too well when tackling difficult combinatorial optimisation problem such as multi-stage nurse rostering. This is because the large action space that need to be explored. This paper proposes to replace the value function approximation by a genetic algorithm in order to generate solutions to the stages before applying the lookahead policy to evaluate the future effect of decisions made in previous stages. Then, the paper proposes a hybrid approach that generates sets of weekly rosters through a genetic algorithm for consideration by the lookahead procedure that assembles a solution for the whole planning horizon of several weeks. Results indicate that this hybrid between an evolutionary algorithm and the lookahead policy mechanism from dynamic programming performs more competitive than the value function approximation dynamic programming investigated before. Results also show that the proposed algorithm is ranked well in respect of several other algorithms applied to the same set of problem instances. The intended contribution of this paper is towards a better understanding of how to successfully apply dynamic programming mechanisms to tackle difficult combinatorial optimisation problems

Small RNA analysis in Sindbis virus infected human HEK293 cells

In contrast to the defence mechanism of RNA interference (RNAi) in plants and invertebrates, its role in the innate response to virus infection of mammals is a matter of debate. Since RNAi has a well-established role in controlling infection of the alphavirus Sindbis virus (SINV) in insects, we have used this virus to investigate the role of RNAi in SINV infection of human cells

Historical institutionalism and the politics of sustainable energy transitions: a research agenda

Improving the understanding of the politics of sustainable energy transitions has become a major focus for research. This paper builds on recent interest in institutionalist approaches to consider in some depth the agenda arising from a historical institutionalist perspective on such transitions. It is argued that historical institutionalism is a valuable complement to socio-technical systems approaches, offering tools for the explicit analysis of institutional dynamics that are present but implicit in the latter framework, opening up new questions and providing useful empirical material relevant for the study of the wider political contexts within which transitions are emerging. Deploying a number of core concepts including veto players, power, unintended consequences, and positive and negative feedback in a variety of ways, the paper explores research agendas in two broad areas: understanding diversity in transition outcomes in terms of the effects of different institutional arrangements, and the understanding of transitions in terms of institutional development and change. A range of issues are explored, including: the roles of electoral and political institutions, regulatory agencies, the creation of politically credible commitment to transition policies, power and incumbency, institutional systems and varieties of capitalism, sources of regime stability and instability, policy feedback effects, and types of gradual institutional change. The paper concludes with some observations on the potential and limitations of historical institutionalism, and briefly considers the question of whether there may be specific institutional configurations that would facilitate more rapid sustainable energy transitions